dual-attention

Awni00's group workspace

Group: Abstractor - L=2, d=128, h=8

1-5

of 5

Tags

Notes

Author

awni00

State

Finished

Start time

August 2nd, 2024 4:51:44 PM

Runtime

9h 48m 26s

Tracked hours

Run path

dual-attention/dual_attention--math--algebra__sequence_next_term/9z18jxwl

Linux-4.18.0-477.51.1.el8_8.x86_64-x86_64-with-glibc2.28

Python version

3.11.7

Git repository

git clone https://www.github.com/awni00/abstract_transformer

Git state

git checkout -b "driven-sky-10" 9ce16be9a1cd1fba943e52654fc94c965461f2e0

Command

/gpfs/radev/project/lafferty/ma2393/abstract_transformer/experiments/math/train_abstractor_model.py --task algebra__sequence_next_term --n_epochs 100 --batch_size 512 --d_model 128 --dff 256 --symbol_type symbolic_attention --n_layers 2 --n_heads 8

System Hardware

CPU count	32
Logical CPU count	32
GPU count	1
GPU type	NVIDIA A40

W&B CLI Version

0.17.5

Group

Abstractor - L=2, d=128, h=8

Config parameters are your model's inputs. Learn more

▶
Config parameters:{} 12 keys
- ▶
  abstractor_kwargs:{} 12 keys
- d_model:
  128
- ▶
  decoder_kwargs:{} 7 keys
- ▶
  encoder_kwargs:{} 6 keys
- group:
  "Abstractor - L=2, d=128, h=8"
- in_block_size:
  161
- ▶
  input_spec:{} 2 keys
  - type:
    "token"
  - vocab_size:
    85
- n_layers_dec:
  2
- n_layers_enc:
  2
- out_block_size:
  31
- out_dim:
  85
- ▶
  output_spec:{} 2 keys
  - type:
    "token"
  - vocab_size:
    85

Summary metrics are your model's outputs. Learn more

▶
Summary metrics:{} 10 keys
- epoch:
  99
- extrapolate_loss:
  0.03653674200177193
- extrapolate_perplexity:
  1.1599539518356323
- extrapolate_teacher_forcing_acc:
  0.9533318281173706
- interpolate_loss:
  0.03653674200177193
- interpolate_perplexity:
  1.1599539518356323
- interpolate_teacher_forcing_acc:
  0.9533318281173706
- train_loss:
  0.0160044115036726
- train_teacher_forcing_acc:
  0.9698988795280457
- trainer/global_step:
  390,599

This run produced these artifacts as outputs. Total: 1. Learn more

wandb-history

run-9z18jxwl-history:v0